In [1]:
%reload_ext autotime
import pandas as pd
from bertopic import BERTopic
from bertopic.representation import KeyBERTInspired
import plotly.io as pio
import plotly.express as px # Plotting
pio.renderers.default = 'notebook'
pd.options.plotting.backend = "plotly"
pd.set_option("display.max_columns", None)
pd.set_option("display.max_colwidth", 100)
df = pd.read_excel("givealittle_health.xlsx")
df.index = pd.to_datetime(df["start"])
df["text"] = df.title.str.cat(df[["pitch", "description", "use_of_funds"]].astype(str), sep=" ")
df
✔️ 9.07 s (2024-11-05T14:28:51/2024-11-05T14:29:00)
Out[1]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | ||||||||||||||||||||||
| 2024-11-03 | Young mother of 3 recently diagnosed with aggressive breast cancer | Young mother of three fighting aggresive breast cancer. | https://static.givealittle.co.nz/assets/hero/e582c9fe-dee8-4746-91bf-b21d00ddf11e-320:False | https://givealittle.co.nz/cause/young-mother-of-3-recently-diagnosed | 3900.00 | 0.0 | 181 days to go | Donate | https://givealittle.co.nz/donate/cause/young-mother-of-3-recently-diagnosed | 3 Nov 2024 | 3 May 2025 | 0 | Pirimai, Napier | 38.0 | Imagine facing one of life's biggest battles while also caring for a young family.\nOur beloved ... | Travel costs for treatment.Financial help with household expenses.Cost to helping fix there vehi... | Created by_x000D_\nDebra Evans \n\n\n\n_x000D_\n Paying to a verified bank account of_x000... | Page Moderated | NaN | NaN | NaN | Young mother of 3 recently diagnosed with aggressive breast cancer Young mother of three fightin... |
| 2024-11-01 | Finlee Bremners Road to recovery | Helping our great friends navigate Finlee's long journey with kidney disease. | https://static.givealittle.co.nz/assets/hero/21b5840d-540f-49c1-bac0-b21800abc236-320:False | https://givealittle.co.nz/cause/finlee-bremners-road-to-recovery | 4500.00 | 0.0 | 177 days to go | Donate | https://givealittle.co.nz/donate/cause/finlee-bremners-road-to-recovery | 1 Nov 2024 | 29 Apr 2025 | 0 | Alexandra, Otago | 44.0 | On the 2nd October 2024, Kayla, Michael and Arlo welcomed a beautiful baby girl into the world, ... | All money received will go towards on going costs while Kayla and Michael are away from home wit... | Created by_x000D_\nKelsey McLean \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Finlee Bremners Road to recovery Helping our great friends navigate Finlee's long journey with k... |
| 2024-10-31 | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help | This page has been set up to help Paula access unfunded drugs to offer hope and relieve his symp... | https://static.givealittle.co.nz/assets/hero/bd8b1741-7704-41df-bc4d-b21a0105b9dc-320:False | https://givealittle.co.nz/cause/paulas-stage-4-brain-cancer-fight-please-help | 47522.15 | 55000.0 | 27 days to go | Donate | https://givealittle.co.nz/donate/cause/paulas-stage-4-brain-cancer-fight-please-help | 31 Oct 2024 | 30 Nov 2024 | 0 | Manawatu / Whanganui | 628.0 | Paula is an extraordinary 18 year old young man whose spirit shines brightly even in adversity.\... | Providing the potentially life extending but unfunded drug (Avastin) and to help with the cost o... | Created by_x000D_\nSheree Turia \n\n\n\n_x000D_\n Paying to a verified bank account of_x00... | Page Moderated | 2.0 | Nearly there!\n3 November 2024\nWe are just about at $47,000!!! Thank you all again for your gen... | NaN | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help This page has been set up to hel... |
| 2024-10-31 | Our Sweet boy Jairo | Please help Jairo get the testing and screening that he needs | https://static.givealittle.co.nz/assets/hero/2f49c6b1-267a-4422-8fe8-b21a00cc4779-320:False | https://givealittle.co.nz/cause/our-sweet-boy-jairo | 925.00 | 1000.0 | 19 days to go | Donate | https://givealittle.co.nz/donate/cause/our-sweet-boy-jairo | 31 Oct 2024 | 22 Nov 2024 | 0 | Palmerston North, Manawatu-Wanganui | 9.0 | My 8 week old baby Jairo has had chronic collic and constipation since he was born. I believe he... | It will be used for the consult and further testing as well as petrol to get to Wellington and back | Created by, and paying to a verified bank account of,_x000D_\nDanni Gillett on behalf of My... | Page Moderated | 1.0 | Follow up appointments\n2 November 2024\nJust updating the goal for follow up appointments x\nSh... | NaN | Our Sweet boy Jairo Please help Jairo get the testing and screening that he needs My 8 week old ... |
| 2024-10-30 | Em’s MS Journey | Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. | https://static.givealittle.co.nz/assets/hero/68655bdf-a7bf-4e0c-a9ab-b2190184b42a-320:False | https://givealittle.co.nz/cause/ems-ms-journey | 9640.00 | 90000.0 | 178 days to go | Donate | https://givealittle.co.nz/donate/cause/ems-ms-journey | 30 Oct 2024 | 30 Apr 2025 | 0 | Dunedin, Otago | 136.0 | My daughter is 25 and has recently been diagnosed with Multiple Sclerosis. Em was working fullti... | Flights to Mexico and operation for Stemcell Replacement/ Chemo/ medication. The operation invol... | Created by, and paying to a verified bank account of,_x000D_\nDiane Stanley on behalf of Em... | Page Moderated | NaN | NaN | NaN | Em’s MS Journey Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. My daughter is... |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2016-05-02 | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | Raising money for the little McSweeney family who have spent the last 5 months in hospital! | https://static.givealittle.co.nz/assets/hero/a07c15e8-a593-48bb-a652-a5fa009e35b9-320:False | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 4834.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 2 May 2016 | 6 Jun 2016 | 0 | Canterbury | 122.0 | PLEASE READ THE UPDATE TO THIS POST UNDER UPDATE TAB. We need you all to know how much of an imp... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Elise Hartl... | Page Moderated | 1.0 | Attention all followers of our story\n31 May 2016\nHello, my name is Elise Hartley and I am the ... | NaN | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... |
| 2016-05-01 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | Since February '16 Tarnz has undergone two operations; one removing 34 lymph nodes from her righ... | https://static.givealittle.co.nz/assets/hero/652566e9-6f90-4632-9b3f-a5f900e9af2c-320:False | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1550.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1 May 2016 | 22 Jul 2016 | 0 | Hawke's Bay | 34.0 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with Grad... | NaN | Created by_x000D_\n _x000D_\n Nancy Easthope_x000D_\n \n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... |
| 2016-05-01 | Cardiology Fundraiser | Fundraising for patients & families affected by Heart Disease. | https://static.givealittle.co.nz/assets/hero/bfefeb94-f041-478f-80a0-a5f900c0dc47-320:False | https://givealittle.co.nz/cause/fund4aheart | 915.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fund4aheart | 1 May 2016 | 15 Aug 2016 | 0 | Waikato | 9.0 | We aim to provide clients with the most up-to-date education around their Heart Disease diagnosi... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Cardiology ... | Page Moderated | NaN | NaN | NaN | Cardiology Fundraiser Fundraising for patients & families affected by Heart Disease. We aim to p... |
| 2016-04-30 | Help for Julia, while she is caring for Ian Robertson | Please support me so I'll be able to continue caring for my dear friend Ian Roberston. He suffer... | https://static.givealittle.co.nz/assets/hero/531d73ca-22d8-423b-93be-a5f700a2c8ed-320:False | https://givealittle.co.nz/cause/help4julia | 2645.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/help4julia | 30 Apr 2016 | 30 May 2016 | 0 | Canterbury | 31.0 | I'm caring for my dear Kiwi friend Ian Robertson, who suffers from terminal lung cancer. It's ge... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Julia Schlu... | Page Moderated | NaN | NaN | NaN | Help for Julia, while she is caring for Ian Robertson Please support me so I'll be able to conti... |
| 2016-04-29 | TRUS Biopsy Machine for Taranaki | The men of Taranaki need this TRUS Biopsy Machine to diagnose Prostate Cancer. Reduce waiting t... | https://static.givealittle.co.nz/assets/hero/c9169845-63b5-4c1e-bb50-a5fc00e5abd0-320:False | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 1910.00 | 20000.0 | closed | Read more | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 29 Apr 2016 | 31 Aug 2016 | 0 | Taranaki | 15.0 | Currently the TRUS (Trans Rectal Ultrasound) machine for diagnosing prostate cancer is only avai... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Rotary Club... | Page Moderated | NaN | NaN | NaN | TRUS Biopsy Machine for Taranaki The men of Taranaki need this TRUS Biopsy Machine to diagnose P... |
10418 rows × 22 columns
In [2]:
df.amountRaised.resample("1YE").sum().plot.bar()
✔️ 380 ms (2024-11-05T14:29:00/2024-11-05T14:29:01)
In [3]:
df.resample("ME").size().plot()
✔️ 59.9 ms (2024-11-05T14:29:01/2024-11-05T14:29:01)
In [4]:
keywords = ["covid", "cancer", "leukemia", "heart", "stroke", "kidney"]
for word in keywords:
df[word] = df.text.str.lower().str.count(rf"\b{word}\b")
df
✔️ 1.86 s (2024-11-05T14:29:01/2024-11-05T14:29:03)
Out[4]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | covid | cancer | leukemia | heart | stroke | kidney | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | ||||||||||||||||||||||||||||
| 2024-11-03 | Young mother of 3 recently diagnosed with aggressive breast cancer | Young mother of three fighting aggresive breast cancer. | https://static.givealittle.co.nz/assets/hero/e582c9fe-dee8-4746-91bf-b21d00ddf11e-320:False | https://givealittle.co.nz/cause/young-mother-of-3-recently-diagnosed | 3900.00 | 0.0 | 181 days to go | Donate | https://givealittle.co.nz/donate/cause/young-mother-of-3-recently-diagnosed | 3 Nov 2024 | 3 May 2025 | 0 | Pirimai, Napier | 38.0 | Imagine facing one of life's biggest battles while also caring for a young family.\nOur beloved ... | Travel costs for treatment.Financial help with household expenses.Cost to helping fix there vehi... | Created by_x000D_\nDebra Evans \n\n\n\n_x000D_\n Paying to a verified bank account of_x000... | Page Moderated | NaN | NaN | NaN | Young mother of 3 recently diagnosed with aggressive breast cancer Young mother of three fightin... | 0 | 3 | 0 | 0 | 0 | 0 |
| 2024-11-01 | Finlee Bremners Road to recovery | Helping our great friends navigate Finlee's long journey with kidney disease. | https://static.givealittle.co.nz/assets/hero/21b5840d-540f-49c1-bac0-b21800abc236-320:False | https://givealittle.co.nz/cause/finlee-bremners-road-to-recovery | 4500.00 | 0.0 | 177 days to go | Donate | https://givealittle.co.nz/donate/cause/finlee-bremners-road-to-recovery | 1 Nov 2024 | 29 Apr 2025 | 0 | Alexandra, Otago | 44.0 | On the 2nd October 2024, Kayla, Michael and Arlo welcomed a beautiful baby girl into the world, ... | All money received will go towards on going costs while Kayla and Michael are away from home wit... | Created by_x000D_\nKelsey McLean \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Finlee Bremners Road to recovery Helping our great friends navigate Finlee's long journey with k... | 0 | 0 | 0 | 0 | 0 | 1 |
| 2024-10-31 | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help | This page has been set up to help Paula access unfunded drugs to offer hope and relieve his symp... | https://static.givealittle.co.nz/assets/hero/bd8b1741-7704-41df-bc4d-b21a0105b9dc-320:False | https://givealittle.co.nz/cause/paulas-stage-4-brain-cancer-fight-please-help | 47522.15 | 55000.0 | 27 days to go | Donate | https://givealittle.co.nz/donate/cause/paulas-stage-4-brain-cancer-fight-please-help | 31 Oct 2024 | 30 Nov 2024 | 0 | Manawatu / Whanganui | 628.0 | Paula is an extraordinary 18 year old young man whose spirit shines brightly even in adversity.\... | Providing the potentially life extending but unfunded drug (Avastin) and to help with the cost o... | Created by_x000D_\nSheree Turia \n\n\n\n_x000D_\n Paying to a verified bank account of_x00... | Page Moderated | 2.0 | Nearly there!\n3 November 2024\nWe are just about at $47,000!!! Thank you all again for your gen... | NaN | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help This page has been set up to hel... | 0 | 3 | 0 | 0 | 0 | 0 |
| 2024-10-31 | Our Sweet boy Jairo | Please help Jairo get the testing and screening that he needs | https://static.givealittle.co.nz/assets/hero/2f49c6b1-267a-4422-8fe8-b21a00cc4779-320:False | https://givealittle.co.nz/cause/our-sweet-boy-jairo | 925.00 | 1000.0 | 19 days to go | Donate | https://givealittle.co.nz/donate/cause/our-sweet-boy-jairo | 31 Oct 2024 | 22 Nov 2024 | 0 | Palmerston North, Manawatu-Wanganui | 9.0 | My 8 week old baby Jairo has had chronic collic and constipation since he was born. I believe he... | It will be used for the consult and further testing as well as petrol to get to Wellington and back | Created by, and paying to a verified bank account of,_x000D_\nDanni Gillett on behalf of My... | Page Moderated | 1.0 | Follow up appointments\n2 November 2024\nJust updating the goal for follow up appointments x\nSh... | NaN | Our Sweet boy Jairo Please help Jairo get the testing and screening that he needs My 8 week old ... | 0 | 0 | 0 | 0 | 0 | 0 |
| 2024-10-30 | Em’s MS Journey | Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. | https://static.givealittle.co.nz/assets/hero/68655bdf-a7bf-4e0c-a9ab-b2190184b42a-320:False | https://givealittle.co.nz/cause/ems-ms-journey | 9640.00 | 90000.0 | 178 days to go | Donate | https://givealittle.co.nz/donate/cause/ems-ms-journey | 30 Oct 2024 | 30 Apr 2025 | 0 | Dunedin, Otago | 136.0 | My daughter is 25 and has recently been diagnosed with Multiple Sclerosis. Em was working fullti... | Flights to Mexico and operation for Stemcell Replacement/ Chemo/ medication. The operation invol... | Created by, and paying to a verified bank account of,_x000D_\nDiane Stanley on behalf of Em... | Page Moderated | NaN | NaN | NaN | Em’s MS Journey Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. My daughter is... | 0 | 0 | 0 | 0 | 0 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2016-05-02 | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | Raising money for the little McSweeney family who have spent the last 5 months in hospital! | https://static.givealittle.co.nz/assets/hero/a07c15e8-a593-48bb-a652-a5fa009e35b9-320:False | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 4834.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 2 May 2016 | 6 Jun 2016 | 0 | Canterbury | 122.0 | PLEASE READ THE UPDATE TO THIS POST UNDER UPDATE TAB. We need you all to know how much of an imp... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Elise Hartl... | Page Moderated | 1.0 | Attention all followers of our story\n31 May 2016\nHello, my name is Elise Hartley and I am the ... | NaN | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | 0 | 0 | 0 | 0 | 0 | 0 |
| 2016-05-01 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | Since February '16 Tarnz has undergone two operations; one removing 34 lymph nodes from her righ... | https://static.givealittle.co.nz/assets/hero/652566e9-6f90-4632-9b3f-a5f900e9af2c-320:False | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1550.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1 May 2016 | 22 Jul 2016 | 0 | Hawke's Bay | 34.0 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with Grad... | NaN | Created by_x000D_\n _x000D_\n Nancy Easthope_x000D_\n \n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | 0 | 4 | 0 | 0 | 0 | 0 |
| 2016-05-01 | Cardiology Fundraiser | Fundraising for patients & families affected by Heart Disease. | https://static.givealittle.co.nz/assets/hero/bfefeb94-f041-478f-80a0-a5f900c0dc47-320:False | https://givealittle.co.nz/cause/fund4aheart | 915.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fund4aheart | 1 May 2016 | 15 Aug 2016 | 0 | Waikato | 9.0 | We aim to provide clients with the most up-to-date education around their Heart Disease diagnosi... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Cardiology ... | Page Moderated | NaN | NaN | NaN | Cardiology Fundraiser Fundraising for patients & families affected by Heart Disease. We aim to p... | 0 | 0 | 0 | 5 | 0 | 0 |
| 2016-04-30 | Help for Julia, while she is caring for Ian Robertson | Please support me so I'll be able to continue caring for my dear friend Ian Roberston. He suffer... | https://static.givealittle.co.nz/assets/hero/531d73ca-22d8-423b-93be-a5f700a2c8ed-320:False | https://givealittle.co.nz/cause/help4julia | 2645.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/help4julia | 30 Apr 2016 | 30 May 2016 | 0 | Canterbury | 31.0 | I'm caring for my dear Kiwi friend Ian Robertson, who suffers from terminal lung cancer. It's ge... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Julia Schlu... | Page Moderated | NaN | NaN | NaN | Help for Julia, while she is caring for Ian Robertson Please support me so I'll be able to conti... | 0 | 4 | 0 | 1 | 0 | 0 |
| 2016-04-29 | TRUS Biopsy Machine for Taranaki | The men of Taranaki need this TRUS Biopsy Machine to diagnose Prostate Cancer. Reduce waiting t... | https://static.givealittle.co.nz/assets/hero/c9169845-63b5-4c1e-bb50-a5fc00e5abd0-320:False | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 1910.00 | 20000.0 | closed | Read more | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 29 Apr 2016 | 31 Aug 2016 | 0 | Taranaki | 15.0 | Currently the TRUS (Trans Rectal Ultrasound) machine for diagnosing prostate cancer is only avai... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Rotary Club... | Page Moderated | NaN | NaN | NaN | TRUS Biopsy Machine for Taranaki The men of Taranaki need this TRUS Biopsy Machine to diagnose P... | 0 | 6 | 0 | 0 | 0 | 0 |
10418 rows × 28 columns
In [5]:
df[keywords].describe()
✔️ 15.4 ms (2024-11-05T14:29:03/2024-11-05T14:29:03)
Out[5]:
| covid | cancer | leukemia | heart | stroke | kidney | |
|---|---|---|---|---|---|---|
| count | 10418.000000 | 10418.000000 | 10418.000000 | 10418.000000 | 10418.000000 | 10418.000000 |
| mean | 0.068823 | 1.349203 | 0.041083 | 0.321367 | 0.072663 | 0.111058 |
| std | 0.427711 | 2.349014 | 0.340030 | 1.050700 | 0.532337 | 0.782261 |
| min | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 25% | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 50% | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 75% | 0.000000 | 2.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| max | 9.000000 | 25.000000 | 8.000000 | 15.000000 | 23.000000 | 23.000000 |
In [6]:
df.resample("1M")[keywords].sum().plot(height=800)
✔️ 63 ms (2024-11-05T14:29:03/2024-11-05T14:29:03)
In [7]:
topic_model = BERTopic(representation_model=KeyBERTInspired(), min_topic_size=10, verbose=True)
topic_model.fit(df.text)
topic_model.get_topic_info()
✔️ 34.5 s (2024-11-05T14:29:03/2024-11-05T14:29:38)
2024-11-05 14:29:03,647 - BERTopic - Embedding - Transforming documents to embeddings.
Batches: 0%| | 0/326 [00:00<?, ?it/s]
2024-11-05 14:29:18,409 - BERTopic - Embedding - Completed ✓ 2024-11-05 14:29:18,410 - BERTopic - Dimensionality - Fitting the dimensionality reduction algorithm 2024-11-05 14:29:34,135 - BERTopic - Dimensionality - Completed ✓ 2024-11-05 14:29:34,136 - BERTopic - Cluster - Start clustering the reduced embeddings 2024-11-05 14:29:34,641 - BERTopic - Cluster - Completed ✓ 2024-11-05 14:29:34,645 - BERTopic - Representation - Extracting topics from clusters using representation models. 2024-11-05 14:29:37,719 - BERTopic - Representation - Completed ✓
Out[7]:
| Topic | Count | Name | Representation | Representative_Docs | |
|---|---|---|---|---|---|
| 0 | -1 | 645 | -1_charity_support_families_provide | [charity, support, families, provide, services, health, family, community, care, involvement] | [Suicide Prevention and Awareness We will use the funds donated to run campaigns to raise awaren... |
| 1 | 0 | 8061 | 0_cancer_diagnosed_treatment_surgery | [cancer, diagnosed, treatment, surgery, medical, hospital, support, family, involvement, expenses] | [Help My Mum Fight Cancer I would love for my family to be able to support mum’s recovery witho... |
| 2 | 1 | 212 | 1_charity_depression_health_suicide | [charity, depression, health, suicide, nz, mental, wellbeing, support, zealand, community] | [Steps for Change - 3000km walk of New Zealand for youth obesity and mental health Anyone can do... |
| 3 | 2 | 189 | 2_surgery_mastectomy_fund_chest | [surgery, mastectomy, fund, chest, raise, get, costs, transgender, top, transition] | [Leif's Top Surgery Help Leif with saving up for top surgery! Kia ora! My name is Leif, I'm a tr... |
| 4 | 3 | 137 | 3_shaving_shave_hair_cancer | [shaving, shave, hair, cancer, wigs, charity, donate, donated, hospice, beard] | [Mop Chop for Cancer On the 22nd of May we will be be shaving off our hair for the breast Cancer... |
| 5 | 4 | 126 | 4_dentist_dental_tooth_teeth | [dentist, dental, tooth, teeth, dentures, orthodontic, orthodontist, braces, jaw, smile] | [Dental surgery for Salote & Simi Please help me get my life long need for dental surgery in Ind... |
| 6 | 5 | 124 | 5_charity_mercy_volunteering_volunteers | [charity, mercy, volunteering, volunteers, volunteer, healthcare, ship, nursing, hospital, nurse] | [Spine Health Africa 2019 Mission Trip Spine Health Africa Chiropractors and Physiotherapists ar... |
| 7 | 6 | 84 | 6_weightloss_diet_surgery_bariatric | [weightloss, diet, surgery, bariatric, weight, fat, gastric, health, lose, healthy] | [Spare $5 and Help save my life - weight loss surgery Help to find life changing and saving weig... |
| 8 | 7 | 83 | 7_ivf_fertility_embryo_pregnancy | [ivf, fertility, embryo, pregnancy, pregnant, funding, conceive, iui, miscarriages, funds] | [Our Fertility Journey Through IVF Help us make our dream come true and expand our family. Ciara... |
| 9 | 8 | 80 | 8_diabetes_insulin_diabetic_glucose | [diabetes, insulin, diabetic, glucose, sugar, sugars, pancreas, diagnosed, dexcom, cgms] | [Emma Needs a CGM to Help her Type One Diabetes! Hi, My Name is Emma, Im a Type one diabetic. I... |
| 10 | 9 | 73 | 9_cataract_cataracts_glaucoma_eyesight | [cataract, cataracts, glaucoma, eyesight, corneal, lenses, eye, surgeries, glasses, eyes] | [Help Gaelene to Save Her Eyesight I have watched my sister fight many battles and have been pow... |
| 11 | 10 | 70 | 10_endometriosis_hysterectomy_uterus_laparoscopy | [endometriosis, hysterectomy, uterus, laparoscopy, ovaries, pelvic, surgery, cyst, endo, menopause] | [Help me live a better quality of life Help me to reach my goal of surgery with Dr Michael East,... |
| 12 | 11 | 65 | 11_hearing_cochlear_deaf_ears | [hearing, cochlear, deaf, ears, ear, hear, aid, costs, funding, cost] | [30 years of hearing loss. After living the last 30 years half deaf, I am ready for a life chan... |
| 13 | 12 | 62 | 12_fiji_fijian_tonga_villages | [fiji, fijian, tonga, villages, village, volunteering, volunteer, supplies, nonprofit, villagers] | [Fiji Village Project Medical screening, health education and restoration of clean water systems... |
| 14 | 13 | 40 | 13_mammaplasty_breast_bra_breasts | [mammaplasty, breast, bra, breasts, surgery, bras, chest, implants, implant, boobs] | [INSECURITY made me Choose THEM but THEY destroyed me..... I was selfish & had Breast Implants ... |
| 15 | 14 | 36 | 14_vanuatu_volunteer_vila_providing | [vanuatu, volunteer, vila, providing, island, supplies, clinic, donation, accommodation, dental] | [Vanuatu Medical Outreach Support the Smith family medical volunteer trip to Vanuatu in April th... |
| 16 | 15 | 33 | 15_defibrillators_defibrillator_defibrillation_aeds | [defibrillators, defibrillator, defibrillation, aeds, aed, ambulance, cpr, cardiac, emergency, d... | [Defibrillator for our community! Every second counts! Help fundraise to be prepared for an emer... |
| 17 | 16 | 33 | 16_alopecia_wig_wigs_hairloss | [alopecia, wig, wigs, hairloss, bald, hair, eyebrows, head, aroha, ash] | [Help towards getting a long term hair solution Raising funds to put towards two custom real hai... |
| 18 | 17 | 29 | 17_hospice_palliative_care_donate | [hospice, palliative, care, donate, memorial, fundraising, patients, dying, providing, provides] | [Te Omanga Hospice 2023 Annual Appeal Te Omanga Hospice 2023 Annual Appeal One in three people w... |
| 19 | 18 | 26 | 18_disability_dog_therapy_dogs | [disability, dog, therapy, dogs, assistance, funds, puppy, pup, costs, service] | [Hoping for a Christmas miracle - (a four-legged, tail-wagging, girls-best-friend miracle!) Hopi... |
| 20 | 19 | 24 | 19_sanitary_periods_donations_tampons | [sanitary, periods, donations, tampons, period, donate, menstrual, poverty, disposable, funds] | [Sanitary Items for Girls and Women in Need (Hawkes Bay) Providing sanitary packs for girls and ... |
| 21 | 20 | 24 | 20_psoriasis_treatment_eczema_dermatologist | [psoriasis, treatment, eczema, dermatologist, hidradenitis, therapy, steroid, skin, medications,... | [TSW Treatment to get my life back Help Grace receive treatment to get her life back! Dear famil... |
| 22 | 21 | 23 | 21_vet_vets_veterinary_donate | [vet, vets, veterinary, donate, funds, pay, rescue, dog, cat, surgery] | [Please help us help our King Kush Please help save our King Kush Kush has a large mass in his a... |
| 23 | 22 | 22 | 22_jaw_surgeries_maxillofacial_surgery | [jaw, surgeries, maxillofacial, surgery, dental, temporomandibular, surgeon, prosthetic, bilater... | [Ashleigh Needs a New Jaw! I am medically requiring a bilateral jaw joint replacement. This is t... |
| 24 | 23 | 20 | 23_vet_vets_kitty_cat | [vet, vets, kitty, cat, surgery, amputation, trixie, dog, funds, her] | [Help save Trixie! Help give Trixie the help she needs! Trixie is a retired working/assistant do... |
| 25 | 24 | 19 | 24_oxygen_breathing_breathe_emphysema | [oxygen, breathing, breathe, emphysema, lungs, lung, portable, transplant, batteries, funds] | [Your help is needed Raise money to buy portable oxygen concentrator Mother of four seeking mo... |
| 26 | 25 | 14 | 25_samoa_samoan_unicef_tonga | [samoa, samoan, unicef, tonga, islands, oceania, vanuatu, island, gecaf, nauru] | [Running for Mental Health in Samoa Running 200+KMs to raise awareness for mental health in supp... |
| 27 | 26 | 13 | 26_adhd_diagnosis_autism_disorder | [adhd, diagnosis, autism, disorder, psychiatrist, assessed, treatment, assessment, medication, s... | [ADHD Diagnosis for my busy brain Raising funds for an appointment with a Psychiatrist specialis... |
| 28 | 27 | 13 | 27_boxing_charity_auckland_charitable | [boxing, charity, auckland, charitable, donations, fundraiser, sporting, suicide, wellington, hi... | [Wellington IT Heavy Hitters 2019 Suicide Prevention (Event) Suicide prevention boxing fundraise... |
| 29 | 28 | 13 | 28_tonsillectomy_surgery_otolaryngology_adenoidectomy | [tonsillectomy, surgery, otolaryngology, adenoidectomy, gastrostomy, hospital, tonsils, throat, ... | [Please help us raise money for Odins grommets, tonsillectomy and adenoidectomy Please help us r... |
| 30 | 29 | 13 | 29_charitable_healing_recovery_fund | [charitable, healing, recovery, fund, spiritual, support, funds, supporting, addiction, addict] | [Help me continue supporting people in need using frequency & compassionate conversations. Help ... |
| 31 | 30 | 12 | 30_sanitation_beyondwater_charity_toilets | [sanitation, beyondwater, charity, toilets, water, kenya, lesotho, wwwbeyondwaterglobal, uganda,... | [Clean Water for Cambodia! Drinking from dirty water sources is the number one cause of illness ... |
In [8]:
topic_model.visualize_topics()
✔️ 2.3 s (2024-11-05T14:29:38/2024-11-05T14:29:40)
In [9]:
topics_over_time = topic_model.topics_over_time(df.text, df.index, nr_bins=10)
topics_over_time
✔️ 13 s (2024-11-05T14:29:40/2024-11-05T14:29:53)
10it [00:12, 1.28s/it]
Out[9]:
| Topic | Words | Frequency | Timestamp | |
|---|---|---|---|---|
| 0 | -1 | families, family, funds, provide, support | 70 | 2016-04-25 21:21:36 |
| 1 | 0 | diagnosed, cancer, treatment, surgery, medical | 671 | 2016-04-25 21:21:36 |
| 2 | 1 | depression, suicide, health, charity, illness | 29 | 2016-04-25 21:21:36 |
| 3 | 2 | transgender, depression, trans, surgery, dysphoria | 2 | 2016-04-25 21:21:36 |
| 4 | 3 | shave, charity, shaving, donated, cancer | 9 | 2016-04-25 21:21:36 |
| ... | ... | ... | ... | ... |
| 278 | 23 | vet, vets, surgery, fund, insurance | 3 | 2023-12-28 00:00:00 |
| 279 | 24 | donate, donated, oxygen, skylah, portable | 2 | 2023-12-28 00:00:00 |
| 280 | 26 | adhd, diagnosis, autism, assessment, diagnostic | 3 | 2023-12-28 00:00:00 |
| 281 | 27 | charity, charitable, boxing, vet, donated | 1 | 2023-12-28 00:00:00 |
| 282 | 28 | nixies, nixie, tonsils, tonsil, surgery | 1 | 2023-12-28 00:00:00 |
283 rows × 4 columns
In [10]:
topic_model.visualize_topics_over_time(topics_over_time, top_n_topics=20, height=600)
⌛ 721 ns (2024-11-05T14:29:53)
In [11]:
df = df[(df.timeLeft == "closed") & (df.goal > 0)]
df["Target completion %"] = df.amountRaised / df.goal * 100
display(df["Target completion %"].describe())
df.sort_values("Target completion %")
✔️ 28.7 ms (2024-11-05T14:29:53/2024-11-05T14:29:53)
count 4118.000000 mean 66.797576 std 799.319922 min 0.000000 25% 6.100000 50% 23.331667 75% 61.820455 max 44000.000000 Name: Target completion %, dtype: float64
Out[11]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | covid | cancer | leukemia | heart | stroke | kidney | Target completion % | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | |||||||||||||||||||||||||||||
| 2024-10-13 | Flights needed NZ-London to help 83 year old mum through a mastectomy | Raising funds to send a caring daughter to London to look after her 83 year old mum following a ... | https://static.givealittle.co.nz/assets/hero/03a34b2c-1401-4460-8ca3-b20800b6f41e-320:False | https://givealittle.co.nz/cause/flights-needed-nz-london-to-help-83-year-old-mum | 0.0 | 3340.0 | closed | Read more | https://givealittle.co.nz/cause/flights-needed-nz-london-to-help-83-year-old-mum | 13 Oct 2024 | 16 Oct 2024 | 0 | NaN | NaN | My mother has breast cancer and at 83 is having a mastectomy on 24th October. She lives alone an... | Just the flights | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Sian Jones | Page Moderated | NaN | NaN | NaN | Flights needed NZ-London to help 83 year old mum through a mastectomy Raising funds to send a ca... | 0 | 1 | 0 | 0 | 0 | 0 | 0.00000 |
| 2024-06-01 | Me and you against the world | Help keep my bro alive ❤️ | https://static.givealittle.co.nz/assets/hero/11e8353e-d3b1-4e38-983f-b18200aeb589-320:False | https://givealittle.co.nz/cause/me-and-you-against-the-world | 0.0 | 5000.0 | closed | Read more | https://givealittle.co.nz/cause/me-and-you-against-the-world | 1 Jun 2024 | 24 Jun 2024 | 0 | Bay of Plenty | NaN | Hi my name is Jess, my younger brother Corey (28 years old) was diagnosed with a 9cm pituitary g... | Medical expenses - to get an appointment with RestoreMe Clinic and subsequently CBD oil (prescri... | Created by_x000D_\nJessica Kelly \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Me and you against the world Help keep my bro alive ❤️ Hi my name is Jess, my younger brother Co... | 0 | 0 | 0 | 1 | 0 | 0 | 0.00000 |
| 2017-03-18 | Filling removal to reverse multiple sclerosis | Silver filling removal to cure m.s. caused by mercury poisoning | https://static.givealittle.co.nz/assets/hero/97b5a6a5-6fb9-432b-9bb0-a73a0110af59-320:False | https://givealittle.co.nz/cause/silveramalgalmscauseautoimmunedisorders | 4.0 | 250000.0 | closed | Read more | https://givealittle.co.nz/cause/silveramalgalmscauseautoimmunedisorders | 18 Mar 2017 | 20 Jun 2017 | 0 | Northland | 3.0 | Old filling removal is dangerous and requires a dentist that really knows how to remove them saf... | Silver filling removal, 2 teeth now need to be ripped as as root canal will harbour bacteria and... | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Sandy Howell | Page Moderated | 1.0 | 28th april\n3 May 2017\nHad two teeth removed as they required root canals which are a death sen... | NaN | Filling removal to reverse multiple sclerosis Silver filling removal to cure m.s. caused by merc... | 0 | 0 | 0 | 1 | 0 | 0 | 0.00160 |
| 2023-05-19 | Charlies life threatening lung complication. | This page is set up to help support my family during this time of extreme hardship we are experi... | https://static.givealittle.co.nz/assets/hero/bf6e7924-c106-4b1e-b4bf-b0070056a009-320:False | https://givealittle.co.nz/cause/charlies-life-threatening-lung-complication | 7.0 | 100000.0 | closed | Read more | https://givealittle.co.nz/cause/charlies-life-threatening-lung-complication | 19 May 2023 | 10 Oct 2023 | 0 | Auckland | 3.0 | Please help our family as our lives have changed due to our sons right lung not functioning prop... | A vehicle, accomadation in a warm home. 2 sons under 3 + new born twins, funds will support us u... | Created by, and paying to a verified bank account of,_x000D_\nRobert Shane Williams | Page Moderated | NaN | NaN | NaN | Charlies life threatening lung complication. This page is set up to help support my family durin... | 0 | 0 | 0 | 0 | 0 | 0 | 0.00700 |
| 2019-03-05 | Help My Father | My Father is diagnosed with Adult CHD. His position is really critical, please help me so that i... | https://static.givealittle.co.nz/assets/hero/761d7080-ddd4-42e0-9dea-aa07014cb211-320:False | https://givealittle.co.nz/cause/help-my-father | 35.0 | 400000.0 | closed | Read more | https://givealittle.co.nz/cause/help-my-father | 5 Mar 2019 | 30 Apr 2019 | 0 | Auckland | 3.0 | Hello everyone,\nMy name is Manisha i am a single mother from India currently living in New Zeal... | For my father treatment | Created by, and paying to a verified bank account of,_x000D_\nmanisha mittal on behalf of M... | Page Moderated | NaN | NaN | NaN | Help My Father My Father is diagnosed with Adult CHD. His position is really critical, please he... | 0 | 0 | 0 | 2 | 0 | 0 | 0.00875 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2017-01-20 | Cancer (Langhan's Cell Histiocytosis) | Raising money for treatment and recovery of being paralyzed and going through chemotherapy and h... | https://static.givealittle.co.nz/assets/hero/7ca74a99-5881-48a7-a7e4-a6fe0052b1bb-320:False | https://givealittle.co.nz/cause/shaunmurphy | 1160.0 | 10.0 | closed | Read more | https://givealittle.co.nz/cause/shaunmurphy | 20 Jan 2017 | 8 Jul 2017 | 0 | Auckland | 27.0 | Hi, I'm Shaun Murphy and I'm 25 years old.\nIf you don't already know my story, 2 years ago I st... | Rehabilitation treatment and any other costs for chemotherapy and physio | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Shaun murphy | Page Moderated | NaN | NaN | NaN | Cancer (Langhan's Cell Histiocytosis) Raising money for treatment and recovery of being paralyze... | 0 | 6 | 0 | 0 | 0 | 0 | 11600.00000 |
| 2017-03-19 | We need to raise funds for Cibele's surgery | Please help raise funds to pay for a tumour removal surgery. Any donation will help. Thank you. | https://static.givealittle.co.nz/assets/hero/f2446f46-19a2-418b-b6fb-a73b014c6d84-320:False | https://givealittle.co.nz/cause/help4cibele | 2482.0 | 20.0 | closed | Read more | https://givealittle.co.nz/cause/help4cibele | 19 Mar 2017 | 1 May 2017 | 0 | Auckland | 36.0 | Cibele has a tumour in her stomach and desperately needs $20.000 to pay for a surgery to remove ... | To pay for a surgery to remove a tumour in her stomach. | Created by_x000D_\n _x000D_\n Carmosina Braga Trancoso_x000D_\n \n\n\n\n\n\n\n_... | Page Moderated | NaN | NaN | NaN | We need to raise funds for Cibele's surgery Please help raise funds to pay for a tumour removal ... | 0 | 0 | 0 | 0 | 0 | 0 | 12410.00000 |
| 2017-03-22 | To raise money to fight for Justice for Baby Jai curtis Blake preventable Still BIrth | Angela and Scott are seeking support with legal fees into the Still Birth of then unborn child t... | https://static.givealittle.co.nz/assets/hero/14ca9772-ba50-4f2d-91b6-a73e01280edc-320:False | https://givealittle.co.nz/cause/supportbabyjai | 277.0 | 2.0 | closed | Read more | https://givealittle.co.nz/cause/supportbabyjai | 22 Mar 2017 | 30 Apr 2017 | 0 | Canterbury | 16.0 | Angela lost her baby in utero at 37 weeks full term after a complicated pregnancy and lots of he... | funds are to fight the justice system for other women preventing this happening to other expecta... | Created by_x000D_\n _x000D_\n April Ritchie_x000D_\n \n\n\n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | To raise money to fight for Justice for Baby Jai curtis Blake preventable Still BIrth Angela and... | 0 | 0 | 0 | 0 | 0 | 0 | 13850.00000 |
| 2017-07-18 | helping barry after a serious hit n run accident | our friend was in a serious hit and run and will be in hospital and after care for some time and... | https://static.givealittle.co.nz/assets/hero/2bfd0a24-b2ab-412f-b8f8-a7b400cce458-320:False | https://givealittle.co.nz/cause/barrystollery | 1450.0 | 10.0 | closed | Read more | https://givealittle.co.nz/cause/barrystollery | 18 Jul 2017 | 30 Sep 2017 | 0 | Bay of Plenty | 18.0 | One of our local motorcyclists was severly injured in a hit and run this week in tauranga he is ... | help barry to get on his feet and back to some kind of normality ideally he will need to save fo... | Created by_x000D_\n _x000D_\n Briar Morreel_x000D_\n \n\n\n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | helping barry after a serious hit n run accident our friend was in a serious hit and run and wil... | 0 | 0 | 0 | 0 | 0 | 0 | 14500.00000 |
| 2022-02-24 | Help the Mason family | William's upcoming surgery/recovery | https://static.givealittle.co.nz/assets/hero/10960d59-c892-40b4-8121-ae4a0142f5fa-320:False | https://givealittle.co.nz/cause/help-the-mason-family | 2200.0 | 5.0 | closed | Read more | https://givealittle.co.nz/cause/help-the-mason-family | 24 Feb 2022 | 25 Jun 2022 | 0 | Upper Hutt, Wellington | 38.0 | Hi my name is William and I'm 38 years old and I had a massive Heart attack on the 2nd of Februa... | Everything. Travel to appointments in lower hutt and Wellington hospital ,Food, rent, petrol, ki... | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n William Mason | Page Moderated | 4.0 | Thanx all\n12 March 2022\nThanx all your support so far , I can't thank you enough. Have got 2 m... | NaN | Help the Mason family William's upcoming surgery/recovery Hi my name is William and I'm 38 years... | 0 | 0 | 0 | 2 | 0 | 0 | 44000.00000 |
4118 rows × 29 columns
In [12]:
top_10 = df.location.value_counts().head(10)
top_10
✔️ 4.93 ms (2024-11-05T14:29:53/2024-11-05T14:29:53)
Out[12]:
location Auckland 1053 Wellington 348 Canterbury 325 Nationwide 231 Waikato 224 Bay of Plenty 184 International 149 Manawatu / Whanganui 124 Christchurch, Canterbury 123 Otago 103 Name: count, dtype: int64
In [13]:
df[df.location.isin(top_10.index)].groupby("location")["Target completion %"].mean().sort_values(ascending=False).plot.bar(title="Mean target completion % by location")
✔️ 34 ms (2024-11-05T14:29:54/2024-11-05T14:29:54)
In [14]:
df[df.location.isin(top_10.index)].groupby("location")["Target completion %"].median().sort_values(ascending=False).plot.bar(title="Median target completion % by location")
✔️ 35.1 ms (2024-11-05T14:29:54/2024-11-05T14:29:54)
In [15]:
fig = px.box(df[df.location.isin(top_10.index)], x="location", y="Target completion %", title="Target completion % by location")
fig.update_yaxes(range=[0, 200])
fig.show()
✔️ 42.4 ms (2024-11-05T14:29:54/2024-11-05T14:29:54)
In [16]:
px.scatter(df, x="n_donors", y="Target completion %", trendline="ols")
✔️ 645 ms (2024-11-05T14:29:54/2024-11-05T14:29:55)
In [17]:
px.scatter(df[(df.n_donors <= 200) & (df["Target completion %"] <= 110)], x="n_donors", y="Target completion %", trendline="ols")
✔️ 39.1 ms (2024-11-05T14:29:55/2024-11-05T14:29:55)
In [18]:
df["duration"] = (pd.to_datetime(df.end) - pd.to_datetime(df.start)).dt.days
px.scatter(df, x="duration", y="Target completion %", trendline="ols")
✔️ 50.5 ms (2024-11-05T14:29:55/2024-11-05T14:29:55)
In [19]:
px.scatter(df[(df.duration <= 100) & (df["Target completion %"] < 200)], x="duration", y="Target completion %", trendline="ols")
✔️ 41.9 ms (2024-11-05T14:29:55/2024-11-05T14:29:55)
In [20]:
import statsmodels.formula.api as smf
nona = df.dropna(subset=["location", "n_donors", "duration"]).rename(columns={"Target completion %": "Target_completion"})
nona["Cancer"] = nona.text.str.contains("Cancer", case=False).astype(int)
nona["Surgery"] = nona.text.str.contains("Surgery", case=False).astype(int)
nona["Auckland"] = nona.location.str.contains("Auckland").astype(int)
nona["Wellington"] = nona.location.str.contains("Wellington").astype(int)
smf.ols(formula="Target_completion ~ Cancer + Surgery + Auckland + Wellington + n_donors + duration", data=nona).fit().summary()
✔️ 127 ms (2024-11-05T14:29:55/2024-11-05T14:29:55)
Out[20]:
| Dep. Variable: | Target_completion | R-squared: | 0.002 |
|---|---|---|---|
| Model: | OLS | Adj. R-squared: | 0.000 |
| Method: | Least Squares | F-statistic: | 1.293 |
| Date: | Tue, 05 Nov 2024 | Prob (F-statistic): | 0.257 |
| Time: | 14:29:55 | Log-Likelihood: | -33248. |
| No. Observations: | 4103 | AIC: | 6.651e+04 |
| Df Residuals: | 4096 | BIC: | 6.656e+04 |
| Df Model: | 6 | ||
| Covariance Type: | nonrobust |
| coef | std err | t | P>|t| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| Intercept | 66.6308 | 22.857 | 2.915 | 0.004 | 21.819 | 111.443 |
| Cancer | -10.2508 | 27.272 | -0.376 | 0.707 | -63.719 | 43.217 |
| Surgery | 16.0472 | 26.837 | 0.598 | 0.550 | -36.568 | 68.662 |
| Auckland | 2.5624 | 27.949 | 0.092 | 0.927 | -52.232 | 57.357 |
| Wellington | 95.0015 | 41.789 | 2.273 | 0.023 | 13.073 | 176.930 |
| n_donors | 0.0373 | 0.051 | 0.728 | 0.467 | -0.063 | 0.138 |
| duration | -0.1175 | 0.090 | -1.312 | 0.189 | -0.293 | 0.058 |
| Omnibus: | 12446.742 | Durbin-Watson: | 2.003 |
|---|---|---|---|
| Prob(Omnibus): | 0.000 | Jarque-Bera (JB): | 879698200.726 |
| Skew: | 44.419 | Prob(JB): | 0.00 |
| Kurtosis: | 2269.671 | Cond. No. | 951. |
Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.